Medical image mis-segmentation region refinement framework based on dynamic graph convolution

In medical image segmentation tasks, it is hard for traditional Convolutional Neural Network (CNN) to capture essential information such as spatial structure and global contextual semantic features since it suffers from a limited receptive field. The deficiency weakens the CNN segmentation performance in the lesion boundary regions. To handle the aforementioned problems, a medical image mis-segmentation region refinement framework based on dynamic graph convolution is proposed to refine the boundary and under-segmentation regions. The proposed framework first employs a lightweight dual-path network to detect the boundaries and nearby regions, which can further obtain potentially misclassified pixels from the coarse segmentation results of the CNN. Then, we construct the pixels into the appropriate graphs by CNN-extracted features. Finally, we design a dynamic residual graph convolutional network to reclassify the graph nodes and generate the final refinement results. We chose UNet and its eight representative improved networks as the basic networks and tested them on the COVID, DSB, and BUSI datasets. Experiments demonstrated that the average Dice of our framework is improved by 1.79%, 2.29%, and 2.24%, the average IoU is improved by 2.30%, 3.53%, and 2.39%, and the Se is improved by 5.08%, 4.78%, and 5.31% respectively. The experimental results prove that the proposed framework has the refinement capability to remarkably strengthen the segmentation result of the basic network. Furthermore, the framework has the advantage of high portability and usability, which can be inserted into the end of mainstream medical image segmentation networks as a plug-and-play enhancement block.

Exogenous Chemicals Impact Virus Receptor Gene Transcription: Insights from Deep Learning.

Despite the fact that coronavirus disease 2019 (COVID-19), caused by severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2), has been disrupting human life and health worldwide since the outbreak in late 2019, the impact of exogenous substance exposure on the viral infection remains unclear. It is well-known that, during viral infection, organism receptors play a significant role in mediating the entry of viruses to enter host cells. A major receptor of SARS-CoV-2 is the angiotensin-converting enzyme 2 (ACE2). This study proposes a deep learning model based on the graph convolutional network (GCN) that enables, for the first time, the prediction of exogenous substances that affect the transcriptional expression of the ACE2 gene. It outperforms other machine learning models, achieving an area under receiver operating characteristic curve (AUROC) of 0.712 and 0.703 on the validation and internal test set, respectively. In addition, quantitative polymerase chain reaction (qPCR) experiments provided additional supporting evidence for indoor air pollutants identified by the GCN model. More broadly, the proposed methodology can be applied to predict the effect of environmental chemicals on the gene transcription of other virus receptors as well. In contrast to typical deep learning models that are of black box nature, we further highlight the interpretability of the proposed GCN model and how it facilitates deeper understanding of gene change at the structural level.

Predicting COVID-19 using lioness optimization algorithm and graph convolution network.

In this paper, a graph convolution network prediction model based on the lioness optimization algorithm (LsOA-GCN) is proposed to predict the cumulative number of confirmed COVID-19 cases in 17 regions of Hubei Province from March 23 to March 29, 2020, according to the transmission characteristics of COVID-19. On the one hand, Spearman correlation analysis with delay days and LsOA are used to capture the dynamic changes of feature information to obtain the temporal features. On the other hand, the graph convolutional network is used to capture the topological structure of the city network, so as to obtain spatial information and finally realize the prediction task. Then, we evaluate this model through performance evaluation indicators and statistical test methods and compare the results of LsOA-GCN with 10 representative prediction methods in the current epidemic prediction study. The experimental results show that the LsOA-GCN prediction model is significantly better than other prediction methods in all indicators and can successfully capture spatio-temporal information from feature data, thereby achieving accurate prediction of epidemic trends in different regions of Hubei Province.

PRISTINE: Semi-supervised Deep Learning Opioid Crisis Detection on Reddit

The drug abuse epidemic has been on the rise in the past few years, particularly after the start of COVID-19 pandemic. Our preliminary observations on Reddit alone show that discussions on drugs from 2018 to 2020 increased between a range of 45% to 200%, and so has the number of unique users participating in those discussions. Existing efforts focused on utilizing social media to distinguish potential drug abuse chats from unharmful chats regardless of what drug is being abused. Others focused on understanding the trends and causes of drug abuse from social media. To this end, we introduce PRISTINE (opioid crisis detection on reddit), our work dynamically detects-and extracts evolving misleading drug names from Reddit comments using reinforced Dynamic Query Expansion (DQE) and constructs a textual Graph Convolutional Network with the aid of powerful pre-trained embeddings to detect which type of drug class a Reddit comment corresponds to. Further, we perform extensive experiments to investigate the effectiveness of our model. © 2022 IEEE.

Multimodal negative sentiment recognition of online public opinion on public health emergencies based on graph convolutional networks and ensemble learning

To improve the effect of multimodal negative sentiment recognition of online public opinion on public health emergencies, we constructed a novel multimodal fine-grained negative sentiment recognition model based on graph convolutional networks (GCN) and ensemble learning. This model comprises BERT and ViT-based multimodal feature representation, GCN-based feature fusion, multiple classifiers, and ensemble learning-based decision fusion. Firstly, the image-text data about COVID-19 is collected from Sina Weibo, and the text and image features are extracted through BERT and ViT, respectively. Secondly, the image-text fused features are generated through GCN in the constructed microblog graph. Finally, AdaBoost is trained to decide the final sentiments recognized by the best classifiers in image, text, and image-text fused features. The results show that the F1-score of this model is 84.13% in sentiment polarity recognition and 82.06% in fine-grained negative sentiment recognition, improved by 4.13% and 7.55% compared to the optimal recognition effect of image-text feature fusion, respectively. © 2023 Elsevier Ltd

HateNet: A Graph Convolutional Network Approach to Hate Speech Detection

The COVID-19 pandemic has caused hate speech on online social networks to become a growing issue in recent years, affecting millions. Our work aims to improve automatic hate speech detection to prevent escalation to hate crimes. The first c hallenge i n h ate s peech r esearch i s t hat e xisting datasets suffer from quite severe class imbalances. The second challenge is the sparsity of information in textual data. The third challenge is the difficulty i n b alancing t he t radeoff b etween utilizing semantic similarity and noisy network language. To combat these challenges, we establish a framework for automatic short text data augmentation by using a semi-supervised hybrid of Substitution Based Augmentation and Dynamic Query Expansion (DQE), which we refer to as SubDQE, to extract more data points from a specific c lass f rom T witter. W e a lso p ropose the HateNet model, which has two main components, a Graph Convolutional Network and a Weighted Drop-Edge. First, we propose a Graph Convolutional Network (GCN) classifier, using a graph constructed from the thresholded cosine similarities between tweet embeddings to provide new insights into how ideas are connected. Second, we propose a weighted Drop-Edge based stochastic regularization technique, which removes edges randomly based on weighted probabilities assigned by the semantic similarities between Tweets. Using 3 different SubDQE-augmented datasets, we compare our HateNet model using eight different tweet embedding methods, six other baseline classification models, and seven other baseline data augmentation techniques previously used in the realm of hate speech detection. Our results show that our proposed HateNet model matches or exceeds the performance of the baseline models, as indicated by the accuracy and F1 score. © 2022 IEEE.

Recent computational drug repositioning strategies against SARS-CoV-2.

Since COVID-19 emerged in 2019, significant levels of suffering and disruption have been caused on a global scale. Although vaccines have become widely used, the virus has shown its potential for evading immunities or acquiring other novel characteristics. Whether current drug treatments are still effective for people infected with Omicron remains unclear. Due to the long development cycles and high expense requirements of de novo drug development, many researchers have turned to consider drug repositioning in the search to find effective treatments for COVID-19. Here, we review such drug repositioning and combination efforts towards providing better handling. For potential drugs under consideration, aspects of both structure and function require attention, with specific categories of sequence, expression, structure, and interaction, the key parameters for investigation. For different data types, we show the corresponding differing drug repositioning methods that have been exploited. As incorporating drug combinations can increase therapeutic efficacy and reduce toxicity, we also review computational strategies to reveal drug combination potential. Taken together, we found that graph theory and neural network were the most used strategy with high potential towards drug repositioning for COVID-19. Integrating different levels of data may further improve the success rate of drug repositioning.

Daily hotel demand forecasting with spatiotemporal features

Purpose: Given the importance of spatial effects in improving the accuracy of hotel demand forecasting, this study aims to introduce price and online rating, two critical factors influencing hotel demand, as external variables into the model, and capture the spatial and temporal correlation of hotel demand within the region. Design/methodology/approach: For high practical implications, the authors conduct the case study in Xiamen, China, where the hotel industry is prosperous. Based on the daily demand data of 118 hotels before and during the COVID-19 period (from January to June 2019 and from January to June 2021), the authors evaluate the prediction performance of the proposed innovative model, that is, a deep learning-based model, incorporating graph convolutional networks (GCN) and gated recurrent units. Findings: The proposed model simultaneously predicts the daily demand of multiple hotels. It effectively captures the spatial-temporal characteristics of hotel demand. In addition, the features, price and online rating of competing hotels can further improve predictive performance. Meanwhile, the robustness of the model is verified by comparing the forecasting results for different periods (during and before the COVID-19 period). Practical implications: From a long-term management perspective, long-term observation of market competitors' rankings and price changes can facilitate timely adjustment of corresponding management measures, especially attention to extremely critical factors affecting forecast demand, such as price. While from a short-term operational perspective, short-term demand forecasting can greatly improve hotel operational efficiency, such as optimizing resource allocation and dynamically adjusting prices. The proposed model not only achieves short-term demand forecasting, but also greatly improves the forecasting accuracy by considering factors related to competitors in the same region. Originality/value: The originalities of the study are as follows. First, this study represents a pioneering attempt to incorporate demand, price and online rating of other hotels into the forecasting model. Second, integrated deep learning models based on GCN and gated recurrent unit complement existing predictive models using historical data in a methodological sense. © 2022, Emerald Publishing Limited.

NSCGCN: A novel deep GCN model to diagnosis COVID-19.

AIM: Corona Virus Disease 2019 (COVID-19) was a lung disease with high mortality and was highly contagious. Early diagnosis of COVID-19 and distinguishing it from pneumonia was beneficial for subsequent treatment. OBJECTIVES: Recently, Graph Convolutional Network (GCN) has driven a significant contribution to disease diagnosis. However, limited by the nature of the graph convolution algorithm, deep GCN has an over-smoothing problem. Most of the current GCN models are shallow neural networks, which do not exceed five layers. Furthermore, the objective of this study is to develop a novel deep GCN model based on the DenseGCN and the pre-trained model of deep Convolutional Neural Network (CNN) to complete the diagnosis of chest X-ray (CXR) images. METHODS: We apply the pre-trained model of deep CNN to perform feature extraction on the data to complete the extraction of pixel-level features in the image. And then, to extract the potential relationship between the obtained features, we propose Neighbourhood Feature Reconstruction Algorithm to reconstruct them into graph-structured data. Finally, we design a deep GCN model that exploits the graph-structured data to diagnose COVID-19 effectively. In the deep GCN model, we propose a Node-Self Convolution Algorithm (NSC) based on feature fusion to construct a deep GCN model called NSCGCN (Node-Self Convolution Graph Convolutional Network). RESULTS: Experiments were carried out on the Computed Tomography (CT) and CXR datasets. The results on the CT dataset confirmed that: compared with the six state-of-the-art (SOTA) shallow GCN models, the accuracy and sensitivity of the proposed NSCGCN had improve 8% as sensitivity (Sen.) = 87.50%, F1 score = 97.37%, precision (Pre.) = 89.10%, accuracy (Acc.) = 97.50%, area under the ROC curve (AUC) = 97.09%. Moreover, the results on the CXR dataset confirmed that: compared with the fourteen SOTA GCN models, sixteen SOTA CNN transfer learning models and eight SOTA COVID-19 diagnosis methods on the COVID-19 dataset. Our proposed method had best performances as Sen. = 96.45%, F1 score = 96.45%, Pre. = 96.61%, Acc. = 96.45%, AUC = 99.22%. CONCLUSION: Our proposed NSCGCN model is effective and performed better than the thirty-eight SOTA methods. Thus, the proposed NSC could help build deep GCN models. Our proposed COVID-19 diagnosis method based on the NSCGCN model could help radiologists detect pneumonia from CXR images and distinguish COVID-19 from Ordinary Pneumonia (OPN). The source code of this work will be publicly available at https://github.com/TangChaosheng/NSCGCN.

Bilateral adaptive graph convolutional network on CT based Covid-19 diagnosis with uncertainty-aware consensus-assisted multiple instance learning.

Coronavirus disease (COVID-19) has caused a worldwide pandemic, putting millions of people's health and lives in jeopardy. Detecting infected patients early on chest computed tomography (CT) is critical in combating COVID-19. Harnessing uncertainty-aware consensus-assisted multiple instance learning (UC-MIL), we propose to diagnose COVID-19 using a new bilateral adaptive graph-based (BA-GCN) model that can use both 2D and 3D discriminative information in 3D CT volumes with arbitrary number of slices. Given the importance of lung segmentation for this task, we have created the largest manual annotation dataset so far with 7,768 slices from COVID-19 patients, and have used it to train a 2D segmentation model to segment the lungs from individual slices and mask the lungs as the regions of interest for the subsequent analyses. We then used the UC-MIL model to estimate the uncertainty of each prediction and the consensus between multiple predictions on each CT slice to automatically select a fixed number of CT slices with reliable predictions for the subsequent model reasoning. Finally, we adaptively constructed a BA-GCN with vertices from different granularity levels (2D and 3D) to aggregate multi-level features for the final diagnosis with the benefits of the graph convolution network's superiority to tackle cross-granularity relationships. Experimental results on three largest COVID-19 CT datasets demonstrated that our model can produce reliable and accurate COVID-19 predictions using CT volumes with any number of slices, which outperforms existing approaches in terms of learning and generalisation ability. To promote reproducible research, we have made the datasets, including the manual annotations and cleaned CT dataset, as well as the implementation code, available at https://doi.org/10.5281/zenodo.6361963.

Heterogeneous deep graph convolutional network with citation relational BERT for COVID-19 inline citation recommendation.

The outbreak of COVID-19 brings almost the biggest explosions of scientific literature ever. Facing such volume literature, it is hard for researches to find desired citation when carrying out COVID-19 related research, especially for junior researchers. This paper presents a novel neural network based method, called citation relational BERT with heterogeneous deep graph convolutional network (CRB-HDGCN), for COVID-19 inline citation recommendation task. The CRB-HDGCN contains two main stages. The first stage is to enhance the representation learning of BERT model for COVID-19 inline citation recommendation task through CRB. To achieve the above goal, an augmented citation sentence corpus, which replaces the citation placeholder with the title of the cited papers, is used to lightly retrain BERT model. In addition, we extract three types of sentence pair according citation relation, and establish sentence prediction tasks to further fine-tune the BERT model. The second stage is to learn effective dense vector of nodes among COVID-19 bibliographic graph through HDGCN. The HDGCN contains four layers which are essentially all sub neural networks. The first layer is initial embedding layer which generates initial input vectors with fixed size through CRB and a multilayer perceptron. The second layer is a heterogeneous graph convolutional layer. In this layer, we expand traditional homogeneous graph convolutional network into heterogeneous by subtly adding heterogeneous nodes and relations. The third layer is a deep attention layer. This layer uses trainable project vectors to reweight the node importance simultaneously according to both node types and convolution layers, which further promotes the performance of learnt node vectors. The last decoder layer recovers the graph structure and let the whole network trainable. The recommendation is finally achieved by integrating the high performance heterogeneous vectors learnt from CRB-HDGCN with the query vectors. We conduct experiments on the CORD-19 and LitCovid datasets. The results show that compared with the second best method CO-Search, CRB-HDGCN improves MAP, MRR, P@100 and R@100 with 21.8%, 22.7%, 37.6% and 21.2% on CORD-19, and 29.1%, 25.9%, 15.3% and 11.3% on LitCovid, respectively.

Machine Learning - Based Diagnosis of Covid-19 using Clinical Data

Coronavirus (COVID-19) is a worldwide pandemic caused by SARS Coronavirus 2. (SARS-CoV-2). The COVID-19 epidemic has put global healthcare systems in jeopardy. This study's purpose is to develop and evaluate an automated COVID-19 infection detection system using machine learning and chest x-ray images. Early diagnosis and treatment may help avert major illness and even death. It is presently the most favoured and accurate approach for COVID-19 diagnosis. X-ray imaging of the chest may be used instead of the rRT-PCR test to look for early COVID-19 symptoms. A new machine learning (ML)-based analytical framework for automated COVID-19 diagnosis is created utilizing chest X-ray pictures of likely patients. The proposed framework for COVID-19 disease diagnosis using X-ray images has a 99 percent accuracy for Covid and a 92 percent accuracy for Non-covid in two-class categorization. The investigation suggests the COVID-19 detection framework is better. © 2022 IEEE.

A Contrastive Graph Convolutional Network for Toe-Tapping Assessment in Parkinson's Disease

One of the common motor symptoms of Parkinson’s disease (PD) is bradykinesia. Automated bradykinesia assessment is critically needed for helping neurologists achieve objective clinical diagnosis and hence provide timely and appropriate medical services. This need has become especially urgent after the outbreak of the coronavirus pandemic in late 2019. Currently, the main factor limiting the accurate assessment is the difficulty of mining the fine-grained discriminative motion features. Therefore, we propose a novel contrastive graph convolutional network for automated and objective toe-tapping assessment, which is one of the most important tests of lower-extremity bradykinesia. Specifically, based on joint sequences extracted from videos, a supervised contrastive learning strategy was followed to cluster together the features of each class, thereby enhancing the specificity of the learnt class-specific features. Subsequently, a multi-stream joint sparse learning mechanism was designed to eliminate potentially similar redundant features of joint position and motion, hence strengthening the discriminability of features extracted from different streams. Finally, a spatial-temporal interaction graph convolutional module was developed to explicitly model remote dependencies across time and space, and hence boost the mining of fine-grained motion features. Comprehensive experimental results demonstrate that this method achieved remarkable classification performance on a clinical video dataset, with an accuracy of 70.04% and an acceptable accuracy of 98.70%. These results obviously outperformed other existing sensor- and video-based methods. The proposed video-based scheme provides a reliable and objective tool for automated quantitative toe-tapping assessment, and is expected to be a viable method for remote medical assessment and diagnosis. IEEE

Graph Convolutional Network-Based Rumor Blocking on Social Networks

Misinformation and rumors can spread rapidly and widely through online social networks, seriously endangering social stability. Therefore, rumor blocking on social networks has become a hot research topic. In the existing research, when users receive two opposing opinions, they tend to believe the one arrives first. In this article, we argue that users will dialectically trust the information based on their own opinions rather than the rule of first-come-first-listen. We propose a confidence-based opinion adoption (CBOA) model, which considers the opinion and confidence according to the traditional linear threshold (LT) model. Based on this model, we propose the directed graph convolutional network (DGCN) method to select the <inline-formula> <tex-math notation="LaTeX">$k$</tex-math> </inline-formula> most influential positive cascade nodes to suppress the propagation of rumors. Finally, we verify our method on four real network datasets. The experimental results show that our method can sufficiently suppress the propagation of rumors and obtains smaller number of rumor nodes than the baseline algorithms. IEEE

Forecasting the COVID-19 Space-time Dynamics in Brazil with Convolutional Graph Neural Networks and Transport Modals

One of the major challenges imposed by the SARS-CoV-2 pandemic is the lack of pattern in which the virus spreads, making it difficult to create effective policies to prevent and tackle the pandemic. Several approaches have been proposed to understand the virus behavior and anticipate its infection and death curves at country ans state levels, thus supporting containment measures. Those initiatives generalize well for general extents and decisions, but they do not predict so well the trajectory of the virus through specific regions, such as municipalities, considering their distinct interconnection profiles. Specially in countries with continental dimensions, like Brazil, too general decisions imply that containment measures are applied either too soon or too late. This study presents a novel scalable alternative to forecast the numbers of case and death by SARS-CoV-2, according to the influence that certain regions exert on others. By exploiting a single-model architecture of graph convolutional networks with recurrent networks, our approach maps the main access routes to municipalities in Brazil using the modals of transport, and processes this information via neural network algorithms to forecast at the municipal level ans for the whole country. We compared the performance in forecasting the pandemic daily numbers with three baseline models using Mean Absolute Error (MAE), Symmetric Mean Absolute Percentage Error (sMAPE) and Normalized Root Mean Square Error (NRMSE) metrics, with the forecasting horizon varying from 1 to 25 days. Results show that the proposed model overcomes the baselines when considering the MAE and NRMSE (p ˂0.01), being specially suitable for forecasts from 14 to 24 days ahead. Author

Detecting Vaccine Skepticism on Twitter Using Heterogeneous Information Networks

Identifying social media users who are skeptical of the COVID-19 vaccine is an important step in understanding and refuting negative stance taking on vaccines. While previous work on Twitter data places individual messages or whole communities as their focus, this paper aims to detect stance at the user level. We develop a system that classifies Dutch Twitter users, incorporating not only the texts that users produce, but also their actions in the form of following and retweeting. These heterogeneous data are modelled in a graph structure. Graph Convolutional Networks are trained to learn whether user nodes belong to the skeptical or non-skeptical group. Results show that all types of information are used by the model, and that especially user biographies, follows and retweets improve the predictions. On a test set of unseen users, performance declines somewhat, which is expected considering these users tweeted less and had fewer connections in the graph on average. To consider multiple degrees of vaccine skepticism, the test set was annotated with more fine-grained labels and the model was repurposed to do multiclass classification. While the model trained on binary labels was unsuited for this additional task, heterogeneous information networks were found useful to both accurately model and visualize complex user behaviors. © 2022, The Author(s), under exclusive license to Springer Nature Switzerland AG.

Graph Convolutional Networks for Predicting State-wise Pandemic Incidence in India

In this paper, we analyze the performance of graph convolutional networks (GCNs) in predicting COVID-19 incidence in states and union territories (UTs) in India as a semi-supervised learning task. By training the model with data from a small number of states whose incidence is known, we analyze the accuracy in predicting incidence levels in the remaining states and UTs in India. We explore the effect of pre-existing factors such as foreign visitor count, senior citizen population and population density of states in predicting spread. To show the robustness of this model, we introduce a novel method to choose states for training that reduces bias through random sampling in five regions that cover India’s geography. We show that GCNs, on average, produce a 9% improvement in accuracy over the best performing non-graph-based model and discuss if the results are feasible for use in a real-world scenario. © 2022 IEEE.

Transparent Aspect-Level Sentiment Analysis Based on Dependency Syntax Analysis and Its Application on COVID-19

Aspect-level sentiment analysis identifies fine-grained emotion for target words. There are three major issues in current models of aspect-level sentiment analysis. First, few models consider the natural language semantic characteristics of the texts. Second, many models consider the location characteristics of the target words, but ignore the relationships among the target words and among the overall sentences. Third, many models lack transparency in data collection, data processing, and results generating in sentiment analysis. In order to resolve these issues, we propose an aspect-level sentiment analysis model that combines a bidirectional Long Short-Term Memory (LSTM) network and a Graph Convolutional Network (GCN) based on Dependency syntax analysis (Bi-LSTM-DGCN). Our model integrates the dependency syntax analysis of the texts, and explicitly considers the natural language semantic characteristics of the texts. It further fuses the target words and overall sentences. Extensive experiments are conducted on four benchmark datasets, i.e., Restaurantl4, Laptop, Restaurantl6, and Twitter. The experimental results demonstrate that our model outperforms other models like Target-Dependent LSTM (TD-LSTM), Attention-based LSTM with Aspect Embedding (ATAE-LSTM), LSTM+SynATT+TarRep and Convolution over a Dependency Tree (CDT). Our model is further applied to aspect-level sentiment analysis on "government" and "lockdown" of 1,658,250 tweets about "#COVID-19" that we collected from March 1, 2020 to July 1, 2020. The experimental results show that Twitter users' positive and negative sentiments fluctuated over time. Through the transparency analysis in data collection, data processing, and results generating, we discuss the reasons for the evolution of users' emotions over time based on the tweets and on our models.

A Self-Supervised Metric Learning Framework for the Arising-from-Chair Assessment of Parkinsonians with Graph Convolutional Networks

The onset and progression of Parkinson’s disease (PD) gradually affect the patient’s motor functions and quality of life. The PD motor symptoms are usually assessed using the Movement Disorder Society-sponsored revision of the Unified Parkinson’s Disease Rating Scale (MDS-UPDRS). Automated MDS-UPDRS assessment has been recently required as an invaluable tool for PD diagnosis and telemedicine, especially with the recent novel coronavirus pandemic outbreak. This paper proposes a novel vision-based method for automated assessment of the arising-from-chair task, which is one of the key MDS-UPDRS components. The proposed method is based on a self-supervised metric learning scheme with a graph convolutional network (SSM-GCN). Specifically, for human skeleton sequences extracted from videos, a self-supervised intra-video quadruplet learning strategy is proposed to construct a metric learning formulation with prior knowledge, for improving the spatial-temporal representations. Afterwards, a vertex-specific convolution operation is designed to achieve effective aggregation of all skeletal joint features, where each joint or feature is weighted differently based on its relative factor of importance. Finally, a graph representation supervised mechanism is developed to maximize the potential consistency between the joint and bone information streams. Experimental results on a clinical dataset demonstrate the superiority of the proposed method over the existing sensor-based methods, with an accuracy of 70.60% and an acceptable accuracy of 98.65%. The analysis of discriminative spatial connections makes our predictions more clinically interpretable. This method can achieve reliable automated PD assessment using only easily-obtainable videos, thus providing an effective tool for real-time PD diagnosis or remote continuous monitoring. IEEE

Learning Discriminant Spatial Features with Deep Graph-based Convolutions for Occluded Face Detection

The use of face masks has become a widespread non-pharmaceutical practice to mitigate the transmission of COVID-19. However, achieving accurate facial detection while people wear masks or similar face occlusions is a major challenge. This paper introduces a model to detect occluded or masked faces based on fused convolutional graphs. This model includes a deep neural architecture with two spatial-based graphs that rely on a set of key facial features. First, a distance graph is used to identify geographical similarity between the facial nodes that represent certain key face parts. Second, a correlation graph is formulated to compute the correlations between every two nodes that represent two different augmented facial modalities. Transfer learning is then performed using a pretrained deep architecture as a baseline to map the semantic information into multiple feature filters. Then, discriminant graph convolutions are constructed based on the fusion of distance and correlation graphs. This model evaluates two tasks of facial detection, which are the binary detection of masked or unmasked faces, and multi-category detection of masked, unmasked, or occluded face with no mask. The experimental results on two benchmarking real-world datasets show that the proposed deep learning model is highly effective with an accuracy of 98% achieved in binary detection. Even with high variance in image occlusions, our proposed model has great promise in detecting and distinguishing between types of facial occlusion with an accuracy of 86% reported in multi-category detection. Author